Skip to content

[rig-sampler] fix: recurse into NullableSchema.inner in assertValidSchema - #33

Merged
pelikhan merged 1 commit into
mainfrom
rig-sampler/13-test-plan-7d1e87ea911c7cc5
Jul 23, 2026
Merged

[rig-sampler] fix: recurse into NullableSchema.inner in assertValidSchema#33
pelikhan merged 1 commit into
mainfrom
rig-sampler/13-test-plan-7d1e87ea911c7cc5

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Sample run: 13-test-plan.ts

Run outcome: The stub runner completed in 12 ms (1 turn, no repair). The agent output schema uses s.enum and s.array(s.object(...)) — both well-structured, no repair loops needed.

Analysis: While reading the harness code after the run I found a gap in assertValidSchema. The function validates composite schemas by recursing into:

  • ArraySchema.items
  • RecordSchema.additionalProperties
  • ObjectSchema.properties

...but it did not recurse into NullableSchema.inner. That means a misconfigured schema like:

s.nullable(s.object({ score: "bad" as any }))

silently passed the construction-time schema check. The invalid nested schema would only surface as a confusing error at invocation time — or, worse, generate a repair loop if the malformed schema was serialised into a prompt and the model tried (and failed) to match it.

Change

Added a nullable branch in assertValidSchema that recurses into schema.inner before the properties branch, making schema validation consistent for all composite schema types:

if ("nullable" in schema && schema.nullable === true) {
  assertValidSchema((schema as NullableSchema).inner, agentName, slot, path);
  return;
}

A new test case in src/rig.test.ts verifies the path reports the correct nested key in the error message (output.score).

All 135 existing tests continue to pass.

Generated by Daily Rig Sampler · 72.1 AIC · ⌖ 7.97 AIC · ⊞ 5.2K ·

assertValidSchema validated array items, record values, and object
property schemas but skipped the inner schema of NullableSchema. A
misconfigured nested schema like s.nullable(s.object({ x: 'bad' as any }))
would silently escape the early construction-time check and only surface
a confusing error at invocation time or, worse, produce a repair loop.

This adds a nullable branch that recurses into schema.inner before the
properties branch, making schema validation consistent for all composite
schema types.

Discovered while running sample 13-test-plan.ts: the sample uses
s.nullable indirectly through s.object with enum fields, and analysing
assertValidSchema revealed the gap.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review July 23, 2026 12:16
@pelikhan
pelikhan merged commit 98a216e into main Jul 23, 2026
2 checks passed
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — the fix is correct and well-tested.

📋 Key Themes & Highlights

Key Themes

  • Root cause addressed: The bug is fixed at the validation layer, not papered over at the call site. All composite schema types (array, record, object, nullable) now recurse uniformly.
  • Regression test present: The new test case directly exercises the previously-uncovered path and asserts the correct error-path string (output.score), making future regressions immediately visible.

Positive Highlights

  • ✅ Placement of the nullable branch before the properties branch is correct — avoids a potential false-fall-through if inner is an ObjectSchema.
  • ✅ The path forwarding (no extra segment for nullable) is consistent with the semantics: nullable is a wrapper, not a path step.
  • ✅ All 135 existing tests continue to pass.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 19 AIC · ⌖ 6.65 AIC · ⊞ 6.3K
Comment /matt to run again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant